Guesswork, Large Deviations, and Shannon Entropy
نویسندگان
چکیده
منابع مشابه
Rényi Entropy , Guesswork Moments and Large Deviations C - E Pfister and WG Sullivan
For a large class of stationary probability measures on AN, where A is a finite alphabet, we compute the specific Rényi entropy of order α and the specific guesswork moments of order β > −1. We show that the specific guesswork moment of order β equals the specific Rényi entropy of order α = 1/(1+ β) multiplied by β. The method is based on energy–entropy estimates suggested by statistical physic...
متن کاملGuesswork is not a substitute for Entropy
Shannon entropy is often considered as a measure of uncertainty. It is commonly believed that entropy is a good measure of how many guesses it will take to correctly guess a single value generated by a source. This belief is not well founded. We summarise some work in this area, explore how this belief may have arisen via the asymptotic equipartition property and outline a hands-on calculation ...
متن کاملShannon Entropy , Renyi Entropy , and Information
This memo contains proofs that the Shannon entropy is the limiting case of both the Renyi entropy and the Tsallis entropy, or information. These results are also confirmed experimentally. We conclude with some general observations on the utility of entropy measures. A brief summary of the origins of the concept of physical entropy are provided in an appendix.
متن کاملLarge deviations for Wigner's law and Voiculescu's non-commutative entropy
We study the spectral measure of Gaussian Wigner's matrices and prove that it satis®es a large deviation principle. We show that the good rate function which governs this principle achieves its minimum value at Wigner's semicircular law, which entails the convergence of the spectral measure to the semicircular law. As a conclusion, we give some further examples of random matrices with spectral ...
متن کاملDistribution-Invariant Risk Measures, Entropy, and Large Deviations
The simulation of distributions of financial positions is an important issue for financial institutions. If risk measures are evaluated for a simulated distribution instead of the model-implied distribution, errors of risk measurements needs to be analyzed. For distribution-invariant risk measures which are continuous on compacts we employ the theory of large deviations to study the probability...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: IEEE Transactions on Information Theory
سال: 2013
ISSN: 0018-9448,1557-9654
DOI: 10.1109/tit.2012.2219036